Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex

Neocortical neurons have thousands of excitatory synapses. It is a mystery how neurons integrate the input from so many synapses and what kind of large-scale network behavior this enables. It has been previously proposed that non-linear properties of dendrites enable neurons to recognize multiple patterns. In this paper we extend this idea by showing that a neuron with several thousand synapses arranged along active dendrites can learn to accurately and robustly recognize hundreds of unique patterns of cellular activity, even in the presence of large amounts of noise and pattern variation. We then propose a neuron model where some of the patterns recognized by a neuron lead to action potentials and define the classic receptive field of the neuron, whereas the majority of the patterns recognized by a neuron act as predictions by slightly depolarizing the neuron without immediately generating an action potential. We then present a network model based on neurons with these properties and show that the network learns a robust model of time-based sequences. Given the similarity of excitatory neurons throughout the neocortex and the importance of sequence memory in inference and behavior, we propose that this form of sequence memory is a universal property of neocortical tissue. We further propose that cellular layers in the neocortex implement variations of the same sequence memory algorithm to achieve different aspects of inference and behavior. The neuron and network models we introduce are robust over a wide range of parameters as long as the network uses a sparse distributed code of cellular activations. The sequence capacity of the network scales linearly with the number of synapses on each neuron. Thus neurons need thousands of synapses to learn the many temporal patterns in sensory stimuli and motor sequences.Comment: Submitted for publicatio

Perpetuation Excellence in Teaching, Leadership, and Learning (PETLL): Pilot Implementation Study

A capstone submitted in partial fulfillment of the requirements for the degree of Doctor of Education in the College of Education at Morehead State University by Henry Webb and Jeff Hawkins on March 15, 2013

The New Normal, Adjuncts and Part-Time Instructors

Department chairs are under increasing pressure from administration to replace departing full-time professors with adjuncts. The chair is faced with a challenging environment that includes navigating personalities, workloads, student perception, varied commitment levels, and meeting accreditation standards. Strategies are discussed as participants share ideas to navigate this complicated issue

Supporting Service-Learning in an Existing Curriculum

This group presentation is interactive and provides a solution-based approach to service-learning. Participants may be involved in a variety of ways such as taking part in small-group activities, role playing, case studies, simulations, problem solving or other hands-on instructional activities and will leave with service-learning ideas for their course(s)

Surveillance of swarms and feral honey bees (Apis melliera) for the presence of American foulbrood (Paenibacillus larvae sub. sp. larvae) spores and their habitat preferences in Western Australia

Honey bees were first transported to Western Australia in 1841 (Barrett 1999) and in the years that followed the first feral honey bee swarms soon appeared in the Western Australian landscape. A brood disease of honey bees, American Foulbrood (AFB) became an economic nuisance in Western Australia by 1899 (Helms 1900) with whole apiaries being destroyed in some localities. It is now an endemic disease found in beekeeping operations world-wide.https://researchlibrary.agric.wa.gov.au/bulletins/1234/thumbnail.jp

Dynamic Control Flow in Large-Scale Machine Learning

Many recent machine learning models rely on fine-grained dynamic control flow for training and inference. In particular, models based on recurrent neural networks and on reinforcement learning depend on recurrence relations, data-dependent conditional execution, and other features that call for dynamic control flow. These applications benefit from the ability to make rapid control-flow decisions across a set of computing devices in a distributed system. For performance, scalability, and expressiveness, a machine learning system must support dynamic control flow in distributed and heterogeneous environments. This paper presents a programming model for distributed machine learning that supports dynamic control flow. We describe the design of the programming model, and its implementation in TensorFlow, a distributed machine learning system. Our approach extends the use of dataflow graphs to represent machine learning models, offering several distinctive features. First, the branches of conditionals and bodies of loops can be partitioned across many machines to run on a set of heterogeneous devices, including CPUs, GPUs, and custom ASICs. Second, programs written in our model support automatic differentiation and distributed gradient computations, which are necessary for training machine learning models that use control flow. Third, our choice of non-strict semantics enables multiple loop iterations to execute in parallel across machines, and to overlap compute and I/O operations. We have done our work in the context of TensorFlow, and it has been used extensively in research and production. We evaluate it using several real-world applications, and demonstrate its performance and scalability.Comment: Appeared in EuroSys 2018. 14 pages, 16 figure

Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells

The neocortex is capable of anticipating the sensory results of movement but the neural mechanisms are poorly understood. In the entorhinal cortex, grid cells represent the location of an animal in its environment, and this location is updated through movement and path integration. In this paper, we propose that sensory neocortex incorporates movement using grid cell-like neurons that represent the location of sensors on an object. We describe a two-layer neural network model that uses cortical grid cells and path integration to robustly learn and recognize objects through movement and predict sensory stimuli after movement. A layer of cells consisting of several grid cell-like modules represents a location in the reference frame of a specific object. Another layer of cells which processes sensory input receives this location input as context and uses it to encode the sensory input in the object’s reference frame. Sensory input causes the network to invoke previously learned locations that are consistent with the input, and motor input causes the network to update those locations. Simulations show that the model can learn hundreds of objects even when object features alone are insufficient for disambiguation. We discuss the relationship of the model to cortical circuitry and suggest that the reciprocal connections between layers 4 and 6 fit the requirements of the model. We propose that the subgranular layers of cortical columns employ grid cell-like mechanisms to represent object specific locations that are updated through movement

The N- or C-terminal domains of DSH-2 can activate the C. elegans Wnt/β-catenin asymmetry pathway

AbstractDishevelleds are modular proteins that lie at the crossroads of divergent Wnt signaling pathways. The DIX domain of dishevelleds modulates a β-catenin destruction complex, and thereby mediates cell fate decisions through differential activation of Tcf transcription factors. The DEP domain of dishevelleds mediates planar polarity of cells within a sheet through regulation of actin modulators. In Caenorhabditis elegans asymmetric cell fate decisions are regulated by asymmetric localization of signaling components in a pathway termed the Wnt/β-catenin asymmetry pathway. Which domain(s) of Disheveled regulate this pathway is unknown. We show that C. elegans embryos from dsh-2(or302) mutant mothers fail to successfully undergo morphogenesis, but transgenes containing either the DIX or the DEP domain of DSH-2 are sufficient to rescue the mutant phenotype. Embryos lacking zygotic function of SYS-1/β-catenin, WRM-1/β-catenin, or POP-1/Tcf show defects similar to dsh-2 mutants, including a loss of asymmetry in some cell fate decisions. Removal of two dishevelleds (dsh-2 and mig-5) leads to a global loss of POP-1 asymmetry, which can be rescued by addition of transgenes containing either the DIX or DEP domain of DSH-2. These results indicate that either the DIX or DEP domain of DSH-2 is capable of activating the Wnt/β-catenin asymmetry pathway and regulating anterior–posterior fate decisions required for proper morphogenesis